Skip to content

Support ADK generate_content LLM spans#145

Merged
krisztianfekete merged 4 commits into
agentevals-dev:mainfrom
erauner12:fix-adk-generate-content-extraction
May 15, 2026
Merged

Support ADK generate_content LLM spans#145
krisztianfekete merged 4 commits into
agentevals-dev:mainfrom
erauner12:fix-adk-generate-content-extraction

Conversation

@erauner12
Copy link
Copy Markdown
Contributor

@erauner12 erauner12 commented May 13, 2026

  • Add shared ADK LLM span recognition for legacy call_llm* spans and ADK-payload-bearing generate_content* spans.
  • Use the shared finder in ADK invocation conversion so invoke_agent* traces without call_llm* spans can still become ADK Invocation objects.
  • Preserve provider-only generate_content* exclusion and add focused extraction/conversion regressions.

Validation

  • uv run pytest -k generate_content — 4 passed, 668 deselected
  • uv run pytest tests/test_extraction.py tests/test_converter.py tests/test_trace_metrics.py tests/test_runner.py — 144 passed
  • Local captured-trace validation against a kagent-garden artifact:
    • num_invocations: 2
    • conversion_warnings: []
    • tool_trajectory_avg_score: 1.0
    • eval_status: PASSED
    • llm_calls: 6, tool_calls: 5

Notes

  • This does not require kagent telemetry changes or a downstream trace-normalization shim.

Fixes #144

Copy link
Copy Markdown
Contributor

@krisztianfekete krisztianfekete left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the PR, @erauner12! I added a couple of comments, but otherwise the direction is great here!

Comment thread src/agentevals/converter.py Outdated
Comment thread src/agentevals/extraction.py Outdated
Comment thread src/agentevals/converter.py
Comment thread tests/test_converter.py
Align ADK LLM span ordering with existing start-time-only extractor conventions.

Make missing-LLM conversion diagnostics name call_llm and ADK generate_content compatible span shapes.

Trim unreachable inner PascalCase ADK content fallbacks while preserving lower/camelCase function call parsing, and add regression coverage for generate_content functionCall content.

Validation: uv run pytest tests/test_extraction.py tests/test_converter.py; uv run pytest -k generate_content
@erauner12
Copy link
Copy Markdown
Contributor Author

Let me know if you have any more suggestions! @krisztianfekete

Copy link
Copy Markdown
Contributor

@krisztianfekete krisztianfekete left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, added one last comment, then we are good to go!

Comment thread src/agentevals/extraction.py
erauner12 added 2 commits May 15, 2026 11:46
Remove unreachable inner Role, Parts, and Text fallbacks from ADK request/response text extraction while preserving outer Contents/Content handling.

Add focused coverage for outer PascalCase ADK payload objects with lower-case inner content fields.

Validation: uv run pytest tests/test_extraction.py -k 'ExtractUserText or ExtractAgentResponse'; uv run pytest tests/test_extraction.py tests/test_converter.py
Format extraction.py to satisfy CI lint.

Validation: uv run ruff check . && uv run ruff format --check .; uv run pytest tests/test_extraction.py tests/test_converter.py
@krisztianfekete krisztianfekete merged commit 9c39e64 into agentevals-dev:main May 15, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ADK converter should extract generate_content LLM spans under invoke_agent

2 participants